Repository navigation
Conversation
|
Commit: Regression test reportSelected 19 changed tests (limit=20, 0 not selected) 46 tests: regression=37 unreleased=8 coverage=1 failing=0
TestAccept/bundle/migrate/auto/registered-model-rename/DMS=: branch ✅ | main (4c8b7a1) ❌ | latest (v1.26.0) ❌main (4c8b7a1): latest (v1.26.0): same output as main (4c8b7a1) TestAccept/bundle/resources/catalogs/rename/DMS=/READPLAN=: branch ✅ | main (4c8b7a1) ❌ | latest (v1.26.0) ❌main (4c8b7a1): latest (v1.26.0): TestAccept/bundle/resources/catalogs/rename/DMS=/READPLAN=1: branch ✅ | main (4c8b7a1) ❌ | latest (v1.26.0) ❌main (4c8b7a1): latest (v1.26.0): TestAccept/bundle/resources/catalogs/rename/DMS=true/READPLAN=: branch ✅ | main (4c8b7a1) ❌ | latest (v1.26.0) ❌main (4c8b7a1): latest (v1.26.0): TestAccept/bundle/resources/catalogs/rename/DMS=true/READPLAN=1: branch ✅ | main (4c8b7a1) ❌ | latest (v1.26.0) ❌main (4c8b7a1): latest (v1.26.0): 40 more failure details omitted, see the full report in the run. |
✅ Integration test reportCommit: 66ba15f
8 interesting tests: 8 flaky
Top 50 slowest tests (at least 2 minutes):
|
344a6c9 to
24b12d9
Compare
24b12d9 to
35b8737
Compare
35b8737 to
5bd3516
Compare
5bd3516 to
a333a4b
Compare
a333a4b to
4ccf5ab
Compare
4ccf5ab to
cdc56bf
Compare
cdc56bf to
0d1ef07
Compare
0d1ef07 to
d16b7c2
Compare
d16b7c2 to
5e37c6e
Compare
5e37c6e to
2957d38
Compare
2957d38 to
ce6daec
Compare
ce6daec to
2b74f07
Compare
simonfaltum
left a comment
There was a problem hiding this comment.
🤖 Automated review from simonfaltum's review agent (pr-review skill) at b667ba5. Each finding was verified against the code; simonfaltum may follow up.
Not approving yet: 1 blocking issue(s), tagged [blocking].
This PR lets a UC secret follow a renamed catalog or schema: it lists the secret's ID fields as provided ID fields and adds DoUpdateWithID. The common case works and is well tested, including on real workspaces, and CI is green. Because the secret's own name is now a provided ID field, a name that references another resource's renamable name is re-keyed in state instead of recreated. State then points at a secret that doesn't exist, which I reproduced on the test server. Two smaller points: a failed update after the parent rename leaves the bundle stuck on 409, and the adapter.go hunk never runs for secrets, so it can be dropped.
| - field: name | ||
| reason: id_field |
There was a problem hiding this comment.
[blocking] Listing the secret's own name in provided_id_fields lets followsParentRename treat any pure reference in name as following a parent rename. parentRenames keys on whatever field holds the reference, and followsParentRename doesn't check which field it is.
Example: name: ${resources.schemas.app.name} in a secret that lives in another schema (schema_name: shared). When app is renamed from app_a to app_b, the plan shows no change for the secret and marks it update_id/parent_renamed with state_only. The deploy records main.shared.app_b, which doesn't exist; UC still has main.shared.app_a. The next plan then wants to create it. If the value or comment also changes in that deploy, DoUpdateWithID PATCHes main.shared.app_b and the deploy fails with 404. I reproduced both against the test server. Before this PR the same config planned a recreate.
Suggested fix: keep name out of the parent-rename path. Either drop it from provided_id_fields (the generated recreate_on_changes still recreates on it), or make followsParentRename consider only the fields that name the parent. Please also add a test with name: ${resources.schemas.x.name}.
Related, already in the base branch: addParentMoves protects a resource's own name only when it is an updatable_id_field. For this same config, the recreate deletes at main.shared.app_b, gets 404, and leaves main.shared.app_a orphaned. The same fix in both places would close that too.
Optionally, DoUpdateWithID could return an internal error when the name in id differs from state.Name. Then a misrouted name change fails clearly instead of PATCHing a path that doesn't exist.
| func (r *ResourceSecret) DoUpdateWithID(ctx context.Context, _ string, state *catalog.Secret, entry *PlanEntry) (string, *catalog.Secret, error) { | ||
| newID := r.IDFromState(state) | ||
| remote, err := r.DoUpdate(ctx, newID, state, entry) | ||
| return newID, remote, err | ||
| } |
There was a problem hiding this comment.
[nice to have] Once the schema's PATCH has moved the secret, a failure of the secret's PATCH at the new path leaves its state at the old ID. Secrets have no internal_id_field, so findMoved cannot find it there. Every later deploy reads the old ID, gets 404, plans a create, and fails with 409 RESOURCE_ALREADY_EXISTS, even after the cause is fixed. Any non-transient error triggers this: a permission error, or an expire_time the backend rejects.
I reproduced it on the test server: rename the schema, change comment, and make the PATCH return 403. Before this PR the same deploy went through recreate, and a failed create dropped the state, so the next deploy could recover. Registered models in #7379 share this gap.
Suggested fix: when an update_id only follows a parent rename, record the new ID before calling DoUpdateWithID, so a failed update is retried at the right path. Another option is to let the plan look at IDFromState when the recorded ID is gone and the parent's rename is already in state. Either way, please add a test.
| // A parent's rename also re-keys a resource through its provided_id_fields (see followsParentRename). | ||
| canUpdateWithID := hasUpdateWithIDTrigger || (a.resourceConfig != nil && len(a.resourceConfig.ProvidedIDFields) > 0) | ||
| if a.doUpdateWithID != nil && !canUpdateWithID { | ||
| return errors.New("DoUpdateWithID is implemented but resourceConfig lacks updatable_id_fields and provided_id_fields") | ||
| } |
There was a problem hiding this comment.
[nice to have] This whole check runs only when a.overrideChangeDesc == nil (line 474). ResourceSecret implements OverrideChangeDesc for value, so the check never ran for secrets, before or after this change. I reverted these lines and go test ./bundle/direct/... plus the secret and rename_with_model_and_secret acceptance tests still pass. Every other resource with DoUpdateWithID and no OverrideChangeDesc already has updatable_id_fields.
So the only effect is that any resource with provided_id_fields (most of them) can now define a DoUpdateWithID that nothing calls. That is what this guard exists to catch, and AGENTS.md asks to avoid speculative flexibility. Please drop the hunk and the matching sentence in the description. If it should stay for future resources, also require IDFromState, as canFollowParentRename does, and add a unit test.
andrewnester
left a comment
There was a problem hiding this comment.
No blocking findings from a full pass; I'll take a manual look at the direct changes before approving.
b667ba5 to
c2907e3
Compare
| # When READPLAN=1, "bundle deploy" is called as "bundle deploy --plan .databricks/plan.json". | ||
| # Normalize so both variants produce identical output. | ||
| [[Repls]] | ||
| Old = ' --plan .databricks/plan.json' |
There was a problem hiding this comment.
isn't it already handled by $(readplanarg)?
There was a problem hiding this comment.
No, readplanarg only expands to --plan .databricks/plan.json (or to nothing). These tests trace the deploy, so with READPLAN=1 the >>> [CLI] bundle deploy --plan .databricks/plan.json line would differ from the other variant; the Repl strips the flag so both variants share one output.txt. The other readplanarg tests do not trace their deploy, so they do not need it.
c2907e3 to
6536422
Compare
fe42ff4 to
1e1984e
Compare
UC moves a secret with its renamed parent, so the planner now re-keys it (update_id, parent_renamed) instead of recreating it, which failed with ALREADY_EXISTS. A secret cannot be renamed itself, so its DoUpdateWithID only re-keys; the adapter check now accepts DoUpdateWithID for a resource whose provided_id_fields a parent rename can change. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
…t; drop duplicated test-server moves Co-authored-by: Isaac <no-reply@databricks.com>
The recreate deletes the secret at its recorded path after the schema's rename moved it, so the moved secret is left behind. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
…secret Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
…rets (review S3) The check is skipped for resources that implement OverrideChangeDesc, as secrets do. Co-authored-by: Isaac <no-reply@databricks.com>
Only parent_id_fields (catalog_name, schema_name, from #7718) follow a parent's rename, so a name that references a renamed schema's name recreates the secret instead of re-keying it to a path that does not exist. Co-authored-by: Isaac <no-reply@databricks.com>
…ADY_EXISTS Verified on AWS, Azure and GCP: "Secret '<full_name>' already exists". Co-authored-by: Isaac <no-reply@databricks.com>
…adness, review G1) Co-authored-by: Isaac <no-reply@databricks.com>
1e1984e to
66ba15f
Compare
Sankalp-Mittal
left a comment
There was a problem hiding this comment.
[AI Generated] Review by review-factory (Claude Opus 5.5 + Claude Fable 5.2, cross-verified). Only points the reviewers agreed on are posted.
| trace $CLI bundle plan | ||
|
|
||
| title "The create fails: the secret is already there" | ||
| trace errcode $CLI bundle deploy |
There was a problem hiding this comment.
[AI Generated] nit: This deploy is expected to fail with 409, and .agents/rules/testing.md asks for musterr rather than errcode when a failure is required. Can this be trace musterr $CLI bundle deploy?
Changes
A UC secret in a renamed catalog or schema (
schema_name: ${resources.schemas.<s>.name}) now follows its parent the way volumes do since #7343 and registered models do in #7379: withIDFromState, the framework re-keys it without a request (state_only). Before, it was recreated where the parent's rename moved it: the secret was deleted and created again.UC cannot rename a secret, so its own name still recreates, even when it references a renamed resource's name: only its
parent_id_fields(catalog_name,schema_name, from #7718) follow a parent. When the secret also has its own changes, the newDoUpdateWithIDsends the update at the new path, and the framework records the new path first (#7379), so a failed update is retried there.Stacked on #7379.
Tests
secrets/recreate_in_renamed_schema(Cloud = true): rename the schema and the secret in one deploy. The secret is recreated, and the old one is deleted where the schema's rename moved it.secrets/parent_rename(Cloud = true): rename the schema (no request to the secret), then rename the catalog and change the secret's comment in one deploy (one PATCH at the new path).create_timestays the same and the final plan is a no-op.secrets/parent_rename_out_of_band(Cloud = true,Badness): a secret has no recorded id, so when its schema is renamed outside the bundle the plan creates it again and UC rejects the create. Same on main.schemas/rename_with_model_and_secret(Cloud = true): the secret now follows the schema's rename too; the deploy sends only the schema's PATCH.Secret '<name>' does not exist.,409 ALREADY_EXISTSSecret '<name>' already exists; checked on AWS, Azure and GCP).This pull request and its description were written by Isaac.